Closed Bug 1271944 Opened 8 years ago Closed 8 years ago

network.predictor.enable-prefetch causes extreme Firefox slowness

Categories

(Core :: Networking, defect)

Product:
Core
Component:
Networking
Version:
48 Branch
Type:
defect
Priority:
Not set
Severity:
major

Tracking

()

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox47 --- unaffected
firefox48 --- disabled
firefox49 - disabled
firefox50 + fixed

People

(Reporter: mr.krzych00, Assigned: u408661)

References

Details

(Keywords: perf, regression, Whiteboard: [necko-active]) 

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:49.0) Gecko/20100101 Firefox/49.0
Build ID: 20160510030240

Steps to reproduce:

Case 1: Surfed using traffic exchanges. 10+ tabs open, constly tab switching.

Case 2: Single tab watching Youtube live stream with chat open and gmail open in 2nd window.


Actual results:

Case 1: Firefox was EXTEMELY SLOW, sites displaying in (i)frames loaded with huge delays. Sometimes they were not being preloaded even after doing full round (getting back to the same tab after finishing all tabs rotation).

Case 2: Stream is constantly buffering at times. After few hours Firefox can take 4GB+ of RAM.


Expected results:

Firefox should load sites almost instantly on my 50mbps / 4mbps VDSL connection as it did in previous versions of nightly (not sure when it got broken).

The cause of this issue is: network.predictor.enabled (100% confirmed for 1st case).

After setting predictor to FALSE Firefox works lightning fast aside from small 0.5-1s lags on tab switching with images being redrawn on zoomed out tabs. YT Live streams with chat also seem to be performing better and take less memory.
Ah one more this I noticed, when network.predictor.enabled is enabled at first it works quite fast. When disk cache gets a lot of entries (20000 ~ 30000 +) it starts to slow down sites being loaded. When I clean disk cache manually in a folder without restarting Firefox, for some time it loads sites faster until it slows down again. However, can't 100% confirm if this is related.
Blocks: 1009122
Component: Untriaged → Networking
Product: Firefox → Core
Severity: normal → major
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(hurley)
Keywords: perf, regression
Version: 49 Branch → 38 Branch
For case 1 firefox seems to be growing in memory usage much slower when predictor is disabled.

I'm not sure how predictor really works. But when it's enabled, can Addons like noscript cause further slowdowns?
It may be also the cause for "addon is making firefox slow" false-positives, but I think I did a lot "ignore permanently" in the past so I'm not able to confirm it atm.
:Virtual - I have no idea why on earth you would think this was 38, the report clearly indicates 49. Please don't change flags when you have no good information to indicate that your change is correct.

I suspect this is related to the leaks that appear to be caused by the prefetch work that landed in 49, which I'm working on in bug 1266671 and Honza is looking at in bug 1267820.

Mr_KrzYch00 - can you try re-enabling the predictor, but disabling prefetch (set network.predictor.enable-prefetch = false) and see if the problem persists? I suspect it won't.
Blocks: 1016628
No longer blocks: 1009122
Flags: needinfo?(hurley) → needinfo?(mr.krzych00)
Version: 38 Branch → 49 Branch
with predictor enabled and predictor's prefetch disabled: no serious issues visible, memory usage pretty stable at 400~600MB with 12 tabs open.
Excellent, thanks. That confirms my suspicions - I'm hoping to make good progress on at least one of the leaks mentioned above today, so I'm taking this and marking it as active while I work the leaks (hopefully fixing those will fix this, and I won't have to circle back around here except to mark this as fixed).

For the record - I haven't seen significant slowness (and my machine isn't exactly super-powerful by modern standards). Doesn't mean it doesn't exist for others, though (obviously)!

You said you are browsing using "traffic exchanges" - do you mean a proxy? VPN? It's not a term I've heard before.
Assignee: nobody → hurley
Depends on: 1267820, 1266671
Summary: network.predictor.enabled causes extreme Firefox slowness → network.predictor.enable-prefetch causes extreme Firefox slowness
Whiteboard: [necko-active]
By traffic exchanges I specifically mean Manual Surfing, it's a program where members are browsing one another's sites by clicking verification image after the timer runs out. Sites are randomly picked by server and are usually shown in [i-]frame, no interaction is required with such advertised site unless it's a verification page to confirm that the member is not using a bot software, otherwise interaction with topbar only occurs (top [i-]frame).

At first it works ok after FF is run, it starts to slowdown after doing few rotation rounds (going in tab-switching circles). When it starts to slowdown a lot of loading bars start appearing in tabs, some sites are starting to have very slow server response, like someone would download a lot of files and no QoS existed on router, meaning lagged connection. However, I can confirm that router was 16 mbps at its peaks and speed test showed 52 mbps total speed. Interesting thing is that some sites were loading pretty ok, just some other lagged so I was also suspecting slow network on the US end (I'm in EU), but after disabling predictor at first (good I saw prefetch there, since I don't need them myself, they usually don't work well for people that use Traffic Exchange systems, because they don't need to download data You actually never want to visit, that's why I disabled predictor as a whole) sites started working almost like on local network.

On a side note: I was about to call my internet provider because of the above (at first I was sure they had some network issues). :)

For case 2 I mentioned. I'm not sure how the prefetch work, but I may suspect it's downloading (or analysing) each member on chat (their name link to their YT channel) making FF cut stream a lot, use 4GB+ memory after having such YT live stream with chat included running for few hours.

I'm not using e10s (so it may be non-e10s bug prefetch?) because of other issues with it. It's actually slower for Manual Surfing due to delays with tabswitching and flashing white page.
(In reply to Nicholas Hurley [:nwgh][:hurley] from comment #4)
> :Virtual - I have no idea why on earth you would think this was 38, the
> report clearly indicates 49. Please don't change flags when you have no good
> information to indicate that your change is correct.

I'm sorry.
I was basing on first comment, mainly on "network.predictor.enabled" setting, which was implanted in Firefox 38.
Status: NEW → ASSIGNED
(In reply to Virtual_ManPL [:Virtual] - (ni? me) from comment #8)
> (In reply to Nicholas Hurley [:nwgh][:hurley] from comment #4)
> > :Virtual - I have no idea why on earth you would think this was 38, the
> > report clearly indicates 49. Please don't change flags when you have no good
> > information to indicate that your change is correct.
> 
> I'm sorry.
> I was basing on first comment, mainly on "network.predictor.enabled"
> setting, which was implanted in Firefox 38.

And I'm sorry for being overly aggressive in my comment. I obviously shouldn't use bugzilla before I've had my coffee in the morning.
The other small thing I found out, when You turn off predictor and don't restart Your browser, some files in cache2\entries are locked, when viewed there is a string "predictor-origin" inside. Shouldn't they be automatically deleted when predictor is turned off?
Depends on: 1271816
I think I'm experiencing this issue as well (or something similar to it).  My page load times seem to be slower, sometimes dramatically slower.  Trying in other browsers seems to be fine for the same sites.  I believe this started happening in nightly in the last week but maybe in the last two weeks at the outside.  It was still happening in today's nightly.

I usually have several hundred tabs open, though the great majority don't ever get viewed between nightly updates.  Immediately on nightly startup, page loads are slow for me.  If it's relevant, I'm running Win7 SP1.

I tried setting network.predictor.enable-prefetch to false and it did immediately seem to help.  I have network.predictor.enable left as true.
Flags: needinfo?(mr.krzych00)
Tracking 50+ based on the performance issue being visible to end users.
tracking-firefox50: ? → +
Wow, the difference between network.predictor.enable-prefetch = false and true is like night and day. I don't use many tabs (on my nightly), and the slowness becomes clearly apparent about 10-15 mins after starting nightly, and after fairly light use (news reading, searches, etc.) At first, I thought it was my ISP, because it felt exactly like when the network turns to mud. All good with enable-prefetch = false.
Developers Edition 49 here, had this problem and disabling prefetch does indeed fix it.
Likewise amazed at how much of a difference turning prefetch off has on the performance of the browser.

I also note that it completely eliminates the problem I had of extremely long Connection times on page assets.  Since I speculated that the long Connection times may be related to a complete failure to load certain pages (detailed in bug #1275917), the issues may be related.
If anyone complaining on this bug can try a recent nightly (built within the last 4 days or so) to see if the problem is fixed, I would appreciate it. There was a patch landed last week for this feature that resulted in some performance wins on our test infrastructure, and I would like to see if it also has the same effect in the real world. Thanks!
Mr_KrzYch00 - see comment 18 - has this behavior improved at all for you with the most recent nightly and network.predictor.enable-prefetch set to true?
Flags: needinfo?(mr.krzych00)
I'm a different user but did seem to be having this problem (comment 11).  For most of yesterday I turned the pref back on.  Compared to when I first commented, it was much better.  But still, sometimes opening new tab would sit with the throbber spinning and a blank white tab.  I'd guess it probably happened 5-10 times over the course of the day.  Today I've turned the pref back off to compare and haven't had it happen yet (the day is about half over by my schedule).
(In reply to Fyren from comment #20)
> sometimes opening new tab would sit with the throbber spinning
> and a blank white tab.  I'd guess it probably happened
> 5-10 times over the course of the day.

This is probably other issue like bug #1275917.
Based on comment 20, the perf win mentioned in comment 18 (see bug 1273882), and the lack of other activity on the bug for the past month, I'm going to call this one fixed.
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Flags: needinfo?(subs)
Flags: needinfo?(mr.krzych00)
Resolution: --- → FIXED
I'm not sure, not really using any prefetches or predictors as they usually are not meant for what I do. I mainly reported it for the sake of others that might've found Firefox sluggish and since I found out what temporarily prevented it I as well shared this information here.

Nonetheless my notebook bricked so I'm now on 1215N netbook (+31% overclocked, 930 CPU Mark) and was forced to switch to Palemoon as Firefox is pretty much unusable on slow devices nowadays, so I can't really test anything anymore until I manage to get myself a new PC.
You need to log in before you can comment on or make changes to this bug.